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ABSTRACT 



The development and use of Oregon's Proficiency-Based 
Admission Standards System (PASS) are described and results of some 
evaluations of the PASS system are presented. PASS is to be used for college 
entry in Oregon and is designed to promote the seamlessness of high school 
and college education by guaranteeing that students would have the knowledge 
and skills needed for college work. The six content areas of PASS are based 
on disciplinary knowledge in mathematics, science, social sciences, foreign 
languages, humanities and literature, and fine and performing arts. 
Criterion-referenced tests, common assessment tasks developed to assess 
cognitively complex proficiencies, and teacher verifications of classroom 
performance will be the basic PASS assessment strategies. Activities are 
under way to set the performance levels for each proficiency and to develop 
the elements of the assessment system. Issues related to the rationale for 
changing the admissions requirements in Oregon are reviewed. These generally 
relate to the relevance of high school preparation for college work and the 
prediction of academic success in college. The PASS project still has 
significant issues to address in the areas of assessment, articulation, 
funding, equity and access, and still must deal with political and 
educational reform issues. Formative evaluation has begun in the original 12 
project schools. Preliminary results with regard to curriculum, instruction, 
assessment, and collaboration are generally favorable, although they reveal 
the amount of work required to make PASS a functioning system statewide. 
(Contains five graphs and one table.) (SLD) 
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Section 1: Historical and Political Context of PASS 



Proficiency-based admissions in Oregon came into being primarily as a response to 
reform legislation aimed at high schools. Legislation passed in 1991 mandated two 
performance-based certificates, the Certificate of Initial Mastery (CIM) and the Certificate of 
Advanced Mastery (CAM). These certificates were to be achieved around ages 16 and 18 
respectively, and were to be benchmarked to "world-class standards." Oregon was to have 
the best-educated students in the nation and the best-prepared workforce in the world. 

The CIM and CAM were to be awarded based on performance, not grades and credits. 
Students would be awarded them only when they met specified standards, not merely after 
spending a designated amount of time in school. This meant students would be applying to 
college with new and unfamiliar transcripts. It also meant these students might have been 
prepared in fundamentally different ways than current applicants. 

The CIM was a general education certificate, and the CAM focused on school-to-work 
transition and career preparation. Neither specifically addressed college preparation or 
admission. The legislation left up in the air many basic issues regarding how high schools 
and colleges would relate to one another. Higher education officials and faculty members 
were suspicious of the CAM in particular, and viewed school reform at best as well- 
intentioned but impractical attempts at social engineering. The fact that the legislation made 
no mention of higher education or its role did not help these perceptions. 

The Oregon State System of Higher Education (OSSHE) was obliged to develop its 
own set of standards as a way to be able to communicate its expectations in the same 
performance-based language as the CIM and CAM. Oregon's seven baccalaureate-granting 
institutions are controlled by one governing board, the State Board of Higher Education, 
which sets admission standards system-wide. 

The State Board of Higher Education (SBHE) had been developing better 
communication with the K-12/community college State Board of Education (SBE) as a result 
of an executive order from the Governor mandating regular joint meetings. The SBHE and 
the SBE met in July, 1993 to discuss how students would move from one system based on 
performance to one based on credits and grades. They agreed that the higher education 
system would provide the K-12 system a list of the knowledge and skills needed for college 
entry. Such a list would facilitate the "seamless" transition between systems for students. 

Seamlessness was another important theme in the state during the early 1990s. 
Legislators and educators alike envisioned a continuous system of learning from preschool 
to postgraduate. Performance standards would be the glue that bound the systems together. 



Dan Underwood, Diana Oxley, and Lynde Paule contributed to this paper 



2 BEST COPY AVAILABLE 



- / ' 



The two boards of education were particularly sensitive to the call for a seamless 
system due to the passage in 1991 of a property tax limitation effort that effectively shifted 
the burden of funding K-12 education from local school districts to the state. The Legislature 
exercised a new-found authority to change education policy in ways that it did not before 
fiscal responsibility was moved to the state level. They were interested in linking the 
systems and began the process through the standards-based certificates. 

Section 2: Structure of PASS 

The Proficiency-based Admission Standards System (PASS) has a number of key 
components. The heart of the system is a clearly-specified description of required knowledge 
and skills. This description is organized into increasing levels of details, beginning with the 
areas of proficiency. 

There are six content areas that form the heart of the system. These are supplemented 
by nine process areas. The six content areas are based on disciplinary knowledge in 
mathematics, science, social sciences, foreign languages, humanities and literature, and fine 
and performing arts. In addition, students must apply intellectual and cognitive skills from 
nine areas as they learn material in the six content areas. The nine process areas are: 
reading, writing, communication competence (speaking, listening), problem solving, critical 
thinking, systems thinking, technology as a learning tool, teamwork, quality work. 

All of the content and process areas have extended definitions which explain the 
boundaries of the area and help signal the key skills that are developed in each area. The 
content areas have more detailed specification in the form of proficiencies. The number of 
proficiencies varies from 5 to 11, with a total of 44 among the 6 content areas. Each 
proficiency is accompanied by a series of indicators to signal more clearly the knowledge and 
skills students will be expected to master. These proficiencies and indicators have the nine 
process areas integrated into them. 

More than 60 national and state curriculum reports were analyzed to determine the 
emerging expectations for American students. The educational systems of other countries 
were examined and, where possible, their standards were analyzed as well. 

The results of these analyses were reviewed by a task force of faculty members from 
Oregon institutions of higher education and educators from high schools and community 
colleges. Their comments and recommendations, in combination with the analysis of the 
reports, served as the basis for writing extended definitions and proficiencies. A subsequent 
task force drafted proficiency indicators. 

Assessment system f 

Three basic assessment strategies will be utilized. 

1. Criterion-referenced tests. Criterion-referenced tests will provide information on content 
knowledge. These tests will often be the same ones required for CIMs and CAMs, allowing 
teachers and students to use this information twice. 

2. Common Assessment Tasks (CATs). CATs will assess cognitively complex proficiencies. 
These tasks will be developed at the state level by trained Oregon educators, university 
faculty, and evaluators, or purchased from other organizations that have created and field 
tested such tasks. The CATs will be scored by teachers trained to scoring criteria that will be 
common statewide. Often, teachers will not score their own students' work. 

3. Teacher verifications. Teachers will certify student performance on those proficiencies 
best judged in the classroom. Teachers will be provided scoring standards and examples of 
acceptable student work for each proficiency they score, and will also identify the means by 
which they reach their determination. 
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Reporting system 

Student performance will be reported in terms of scores on a 1 to 5 scale for each 
proficiency. Scores will be summarized to create a proficiency average. Additionally, plans 
are under way to develop the capacity for students to attach electronically examples of work 
illustrating their mastery of particular proficiencies. The examples will be linked to the 
appropriate proficiency or proficiencies, allowing more in-depth information about a 
student's knowledge when required to make an admission (or placement) decision. 

Oregon students going out of state would have their proficiency average along with a 
description of the required performance level for each of their scores. Out-of-state schools 
will be provided "translations" indicating how the proficiencies translate into traditional 
courses, but will also be encouraged to use the proficiency-based system without translation 
in those states where PASS is currently working to create models of how to use a 
proficiency-based transcript. 

Current development activities 

Activities are underway to set the performance levels for each proficiency and to 
create the required elements of the assessment system. Much of this work is being done in 
coordination with the Oregon Department of Education as it develops the content standards 
and assessments for the CIM and CAM. The goal is to produce one assessment system that 
can be adapted to the needs of the certificates and of higher education admission. 
Proficiency-based admissions will be fully implemented beginning with students accepted 
for admission fall, 2001. In the interim, schools can use various combinations of 
proficiencies and Carnegie units if they receive permission from OSSHE. 

PASS has received four grants, three federal and one private, that enable it to include 
high schools directly as partners in all development activities. Currently, 20 Oregon high 
schools are participating in identifying performance levels for each proficiency and securing 
examples of student work that would meet acceptable performance levels on each criteria. 
These high schools are each linked to an OSSHE institution and a community college. 

These "partnerships" create a vehicle for co-development of proficiency-based admissions. 

Section 3: Rationale and Goals for Change in Admissions 

The primary impetus for changing admission was legislation that changed secondary 
school requirements. However, once OSSHE undertook to develop proficiency standards, it 
became clear that there were a number of reasons to examine new models for admission. 
The following reasons were among those that motivated the adoption of proficiency-based 
admissions: v 

As the state limited property taxes and moved K-12 funding more to the state level, it 
began to decrease funding for higher education. This began gradually, and accelerated 
through the 1990s. The net effect was a cut of approximately 40 percent in state support to 
higher education over a five year period. Legislators called for "increased productivity" 
from the state's colleges and universities. This was translated to mean more hours for 
faculty in front of classes and more students in each classroom. 

Cuts of this magnitude cannot be borne within the system without change. While no 
one in Oregon recommends this as a means to stimulate systems change, it did have the 
effect of expanding the range of options faculty and administration were willing to entertain 
to cope with the cuts. 

In this context, proficiency-based admissions seemed more attractive, since such a 
system could result in increased productivity by having better-prepared students move 



through the system more rapidly. The net effect would be increased productivity; more 
students educated by the same number of faculty. 

Proficiency-based admissions promised help with several other nagging problems. 
OSSHE campuses, like their equivalents in other states, admitted many students who had to 
begin their college studies in remedial courses. OSSHE rates were not exceptionally high 
(approximately 25 percent in writing, 40 percent in math), but did drive productivity and 
standards down as students took longer and longer to achieve the baccalaureate. 

Similarly, completion rates, the proportion of students receiving a degree in six years, 
varied between 23 percent and 54 percent. Such rates are common among state-supported 
schools nationally, but came under increasing scrutiny in a tight fiscal environment. Why 
admit students who could not or did not complete their studies? Why not develop 
admissions methods that selected students likely to take advantage of a state-assisted 
education to receive a degree in four years? 

One other hidden effect of current admission procedures was that the freshman 
curriculum tended to reteach large amounts of the high school curriculum, since college 
faculty could not be certain what their students knew and didn't know. Such practices were 
self defeating in many ways, but most significant because they tended to lower students' 
perceptions of the difficulty of the college program. Such perceptions affected their 
subsequent effort. 

The practice of reteaching the high school curriculum was also difficult to explain to 
legislators. Why was the state paying for a high school education, then paying once again to 
reteach essentially the same things to those students who were supposed to be the best- 
prepared? 

Current admission practices suffered from another problem: grade inflation. 
Requirements to be accepted into the University of Oregon had increased from 2.0 in the 
1960s to 3.0 in the 1990s. Student performance on nationally-normed tests such as the SAT 
remained relatively constant during this period. Students with significantly higher GPAs 
did not seem to know any more when compared with their peers nationally. 

The time-honored method of reviewing each high school's courses and approving 
college preparation based on course titles did little to control content or quality of the 
courses, and even less to create consistency of teacher grading within those courses. The 
approved course list did not communicate to teachers and students what it is they needed to 
be doing within those classes to prepare for college. The proficiencies higher education 
would identify for admission would communicate clearly and unambiguously what 
students needed to be able to know and do to succeed in college, something course titles 
alone could not do. Having the same proficiencies in place statewide allowed teachers to 
design learning experiences that prepared students for college more directly and 
appropriately. 

If Oregon high schools were going to change the way they educated and assessed 
students, colleges would have to be ready. Many high schools had already begun developing 
new curricula for the CIM and CAM. The curricula tended to be more integrated and 
applied. Students would work in teams on projects that had real-world applications. They 
would be assessed via demonstration, exhibition, or portfolio. They would not necessarily 
spend required amounts of time in class, since much of the work required would be done 
independently outside of classs. The Carnegie unit would not capture these integrated 
experiences very well. A proficiency-based model of admission would allow the two systems 
to talk with one another using a common language, the language of performance, not the 
language of time. 
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The remainder of this section examines in turn some of the issues raised earlier in 
this section relative to the rationale for changing admissions criteria. Each subsection 
presents a summary of research conducted by the PASS Project on topics that bear on the 
value, validity, and effects of current grade-based methods of admission. 

Increased Rates of Remediation in Writing and Mathematics among College Freshmen* 

Despite many past efforts undertaken by secondary education to improve student 
readiness for college-level coursework in math and writing, remedial education is an 
increasingly significant issue facing higher education administrators. The rates at which 
first-time freshmen require some remedial education is alarmingly high at most of the 
institutions of higher education in the US. The Oregon State System of Higher Education 
(OSSHE) is no exception — nearly 40% of all first-time freshmen admitted as fully qualified 
students require remedial education in math, and nearly 25% require some remedial 
education in writing (see Graph 1 below). Current college admissions policies are resulting 
in large numbers of "qualified" admitted students in need of remedial education, and may 
compel institutions to reconsider current admission practices. Although it is not always 
immediately clear, post-secondary remedial education has many negative effects on both 
students and the institutions. The purpose of this section is to present some of the negative 
effects of remedial education, particularly in light of the increasing levels or remediation 
required. 

Graph 1 



Remediation Rates for First Time Freshmen at OSSHE Institutions 




Math Writing 



Remedial education is typically viewed in the context of the academic "readiness" of 
first-time freshmen students for first-level college coursework in math and writing. First- 
time freshmen at an OSSHE institution are deemed in need of remedial education if they 
are unable to qualify for placement in College Algebra or above, or in Writing 121 or above. 

Regarding math, OSSHE institutions generally assess a student's readiness by 
requiring a short math test. As the graph above displays, about 40% of the current first-time 
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freshmen in OSSHE institutions do not place in first-level college math (College Algebra). 
Furthermore, although not as substantial as with math, many first-time OSSHE freshmen 
enter the system inadequately prepared for college-level writing as well. About 25% of the 
first time freshmen in OSSHE do not score high enough on placement exams to qualify for 
Writing 121. The long-standing tool used by OSSHE institutions for placement in college- 
level writing is the Test of. Standard Written English (TSWE), with a score of 37 or 38 
usually being the cutoff for admission to Writing 121. 

So, what do these levels of remediation mean to students and institutions in OSSHE? 
"Essentially, it means that remedial education impinges upon both the student and the 
university," says Jim Buch, Vice President for Student Academic Affairs at the University of 
Oregon. "The assumption is that students who perform at satisfactory levels in high school 
courses and on standardized tests will be prepared for first-level college coursework, but 
many times, they aren't." This, he suggests, results in the students paying extra money for 
remedial courses, and puts a university in the difficult position of having to develop a 
means for helping students overcome their inadequate preparation. 

Remedial education can be a very expensive undertaking. For students who do not 
place in first-level college coursework, the effects of engaging in remedial education can be 
substantial. The vast majority of remedial education activity is undertaken at extra costs to 
the participating students, and more often than not, remedial coursework does not carry 
regular college credit (credit that does not satisfy graduation requirements). Although most 
OSSHE institutions do offer some remedial courses, the primary sources of remedial 
education continue to be community colleges and continuing education courses. For 
students, the extra costs associated with taking remedial coursework at community colleges 
or in continuing education programs can be significant, and it is estimated that OSSHE 
students spend at least $300,000 per year on remedial math education alone. When 
comparable figures are added for remedial writing coursework, it is clear that remedial 
education has a significant financial impact on ill-prepared students. It is important to re- 
emphasize that much of this money is spent on coursework that will not be applied toward 
OSSHE graduation requirements. 

In addition to the money that students spend on remedial education, Gary 
Christensen, Assistant Vice Chancellor for Student Services for OSSHE says that unprepared 
students may suffer a more subtle negative effect of remedial education. What is often 
overlooked, Christensen says, is "the effect that remedial education has on a student's 
academic and career choices." Most students entering OSSHE hope to complete their 
programs within four years. But if, for example, a student requires a full year of remedial 
education in math, the student may be limited to academic programs that do not emphasize 
math related skills. "It's sometimes difficult for students to catch-up if they've required 
substantial remedial coursework," Christensen says, "and this may have long-term effects 
on the student because it limits both their academic and career choices." 



Students aren't the only stakeholders to suffer the negative effects of remedial 
education. Both OSSHE as a whole, and the individual institutions within the system feel 
the burden of devoting resources to remedial education. Although, remedial coursework is 
provided through community colleges and continuing education courses, most OSSHE 
institutions offer limited regularly funded remedial coursework that may be applied to 
graduation requirements because of the demand for remedial courses in higher education. 
And in these days of shrinking institutional budgets, remedial course offerings are 
especially costly to an institution. For instance, in 1990-91 OSSHE institutions spent 
approximately $100,000 on instruction costs alone for regularly funded remedial writing 
courses. When this is added to the more expensive remediation costs associated with the 
regularly funded remedial math courses, it becomes clear that colleges and universities are 
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spending a great deal of money on teaching the same skills that are generally taught in high 
schools. 

As with unprepared students, the effects of remedial education on an institution of 
higher education may be subtle, yet significant. Large numbers of unprepared students limit 
an institution's ability to develop and advance their academic programs. Because many 
students aren't ready for first-level college coursework, programs and departments are 
sometimes forced to adjust their curriculums accordingly. And because of the corollary need 
for institutions to offer some remedial education, institutions are often limited in their 
ability to develop the type of curricula and programs that they would like to offer. The net 
result is that remedial education has a far reaching, and very broad negative effect on the 
entire OSSHE community. 

It is clear that not every student entering OSSHE can be adequately prepared for 
college level coursework, and that higher education should have some role in remedial 
education. It is equally clear, however, that keeping remediation rates as low as possible 
serves the best interests of both the students and the institutions. Generally speaking, 
placement in College Algebra and Writing 121 does not pose an unrealistic standard for 
students who have successfully completed a high school curriculum. However, across the 
country there continues to be a large number of first-time freshmen students who require 
some remediation in math and writing. Remediation rates continue to be a concern for 
higher education administrators and faculty, resulting in a growing importance placed upon 
the need to reconsider college admissions in light of the lack of readiness of students for 
post-secondary education. 

In the next section, we discuss why recent trends in high school grade point averages 
may also cause concern for higher education administrators and faculty. 

The Inflation of High School Grade Point Averages' 

For those associated with college admissions, high school grade point averages 
(GPAs) have long been considered one of the strongest indicators of college readiness. But 
does this conventional wisdom still hold true, particularly in light of much recent rhetoric 
concerning so-called "grade inflation?" In this subsection, we consider just how reliable the 
grade point average is as an indicator of college readiness. To do this, we compare GPA with 
SAT scores over the past three decades. 

Although SAT scores have their own problems, they do represent a stable, external 
measure of performance that is explicitly related to college readiness. By comparing the 
GPAs and SAT scores of incoming freshmen over the period 1968-1995, we can ascertain the 
relationship between the teacher-determined grade and the nationally-normed SAT score. If 
grades are reliable, we should expect to see them move roughly in tandem with SAT scores; 
when one increases, so should the other; they should at least trend in the same direction. If 
there is little relationship, we should be concerned about what a GPA is measuring. 

The University of Oregon has used high school GPAs and SATs as a basis for 
admissions decisions since the mid-1960s. The data relating to average GPAs and SATs of 
entering freshmen have been maintained by the UO Office of the Registrar since 1968. 

Graph 2 (below) displays the average SAT scores for freshmen classes entering the 
University between fall 1968 and fall 1995. As Graph 2 shows, the average SAT scores of 
freshmen have dropped from 1026.9 in 1968, to 1010.7 in 1995. In particular, SAT scores 
dropped dramatically in the 1970s. 

Graph 2 
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Conversely, Graph 3 (below) shows the general trend of increasing average GPAs for 
UO freshmen. The freshmen class of 1968 had an average GPA of 3.07, and by 1995 it had 
increased to 3.31. Read together, Graphs 2 and 3 do not show a strong, consistent, or 
predictable relationship between the trend in SAT scores and high school GPA. In fact, the 
correlation is -.07, statistically not significant. The two measures have moved relatively 
independent of one another over time and when considered by cohort . This is not the usual 
way in which this relationship is examined. 



Graph 3 



Average Grade Point Average for Entering Freshmen at the University of Oregon by Year 
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The second question this trend raises is perhaps more disturbing. If GPA has 
increased 1/3 of a point over a 25 year period, how long will it be until GPA reaches a point 
of compression that renders it totally useless as a discriminator? If the average approaches 
3.5, as it soon will if current trends continue, how can admission officers make intelligent 
decisions regarding who should be admitted? They may find themselves with much in 
common with gymnastic meet judges, who have had to resort to scores with three places to 
the right of the decimal point to differentiate athletes, all of whom score between 9 and 10 
on a ten-point scale. This problem is only exacerbated by the predicted increase in demand 
over the next ten years for entry into Oregon's seven public colleges and universities. 

PASS is designed to provide consistent standards statewide that eliminate grade 
inflation. Student work is scored against common criteria. Random review of teacher 
scoring, called "validation," helps ensure high reliability. At the same time, PASS reduces 
reliance on external tests such as the SAT, which, by design, teachers are not supposed to 
prepare students to take. PASS lets teachers and students teach and learn what is necessary 
for entry into and success in college. PASS assessments illustrate the types of skills needed to 
do college-level work successfully. PASS standards inform teachers how to score student 
work against consistent standards. The net effect is a " recenter ing" of the GPA against 
consistent standards and an increase in both its reliability and validity as a tool for 
determining college readiness. We move from "Grade Point Average" to "Proficiency 
Average." 

Similar to high school grades, college freshman grades have inflated over time. The 
next section presents some troubling data regarding the meaningfulness of first-year college 
grades as true indicators of college readiness. 

The Inflation of College Freshmen Grades" 

As with high school grade point averages (GPAs), a predominant assumption among 
educators is that college freshmen grades are strong evidence of college readiness. This 
subsection questions the validity of that assumption by exploring the data available for 
University of Oregon freshmen over the past 25 years. 

Are college freshmen grades reliable indicators of student readiness for college-level 
work? At first glance most would probably answer "yes." After all, if college freshmen 
perform well in their first college courses, this must mean they were ready for college-level 
work. The answer may not be that simple. Graph 4, below, shows the percentage of "A" and 
"B" grades given to freshmen at the University of Oregon during fall terms from 1970 
through 1995 for all first-term, first-year courses has increased substantially over the past 26 
years, from 44% in 1970 to 56% in 1995, a 27% increase overall. 
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Graph 4 



Percentage of "A" and "B" Grades For UO Freshmen 
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During this period SAT scores declined, then gradually increased to reach a level 
slightly below the 1970 freshmen (see Graph 2, above). In other words, students whose SAT 
scores were slightly lower than their peers' 25 years earlier at the University of Oregon were 
receiving A's and B's at almost a 30% greater rate. At the same time, students in all seven 
institutions in the Oregon State System of Higher Education were requiring remediation at 
a rate of approximately 25% in writing (for those who chose to take writing) and 40% in 
mathematics (for those who chose to take mathematics) (see Graph 1, above). 

Because freshmen grades have actually increased during a period when UO freshmen 
do not show any improvements when compared to national populations, we must question 
the reliability of first-year college freshmen grades as indicators of student readiness for 
college level work. First-year freshmen grades may not be particularly reliable indicators of 
student readiness and achievement for a variety of reasons, the most important of which is 
that these courses may be reteaching the high school curriculum (discussed in the next 
section). But whatever its causes may be, the inflation of college freshmen grades seems to 
be eroding their value as indicators of student readiness. 

It appears that neither college freshmen grades nor high school GPAs really are fully 
valid data points to serve as the correlates upon which we base the notion of "predictive 
validity." As long as the assumption is made that a correlation between high school GPA 
and freshman grades validates our admission procedures, colleges will not be able to design 
a college program that can assume much about what incoming students actually know. This 
leads to inefficiency, and worse, complacency and cynicism as students are retaught the 
same things they just learned in high school (albeit perhaps at an accelerated rate) and 
college instructors continually bemoan the poor preparation of each year's crop of freshmen. 

The next subsection discusses the overlap, or redundancy, between high school and 
college level courses that has been alluded to above. Similar to our previous findings, the 
redundancy between high school and college level courses raises questions regarding the 
true college readiness of many students. 



Redundancy in High School and College Level Coursework 

So far we have discussed the limitations of freshmen college grades as reliable 
indicators of student readiness for college level course work. College freshmen grades at the 
University of Oregon (UO), for example, have risen substantially over the past 25 years, 
despite the fluctuating average SAT scores for freshmen entering the UO over the same 
time period. These trends tell us is that the link between college freshmen grades and actual 
student knowledge, skills, and abilities (measured by standardized test scores) is not terribly 
strong — but they don't tell us why. In this section we explore one possible cause for the 
decreasing reliability of college freshmen grades as indicators of student readiness: the 
redundancy in course content between high school and freshmen level courses. 

Part of the rise in college freshmen grades may be attributed to the redundancy, or 
"overlap," between the content of upper level high school coursework and college 
freshmen level coursework. If college freshmen are simply reviewing much of the material 
from their high school courses, we would expect freshmen grades to be relatively high, 
despite declining average SAT scores. To determine the levels of redundancy between what 
is taught and learned at high school and what is taught and learned in college freshmen 
courses, syllabi and course descriptions from four of the most popular college freshmen 
courses were compared to similar high school courses. The courses compared were: College 
Composition I (Writing 121), College Algebra (Math 111), Introductory Biology (Biology 101), 
and First-year Spanish (Spanish 101). All of the information used for the comparison came 
from an Oregon State System of Higher Education institution (OSSHE), and from a large 
Oregon high school. 

Although the levels of redundancy vary somewhat by course, generally, there is 
substantial overlap between the college level and high school level course contents. For 
example, as Table 1 displays (below), the College Composition course (Writing 121) seeks to 
develop students' critical writing skills, with an emphasis on argumentation, persuasion, 
reason, and rationale. Further, College Composition develops synthesis skills around the 
course readings, requiring that students develop a thesis and support it with relevant course 
literature. Finally, College Composition emphasizes process: students are expected to edit 
and revise their work, and to refine grammar, usage, and punctuation in their essays. 

Similarly, high school AP English focuses on many of the same concepts stressed in 
College Composition, including: critical writing skills such as argument, persuasion, 
rationale, and thesis development. Moreover, high school AP English targets the same 
synthesizing skills emphasized in College Composition, including: applying course readings 
in support of a student developed thesis, as well as comparing and contrasting course 
literature. Finally, like College Composition, high school AP English is anchored in the 
writing process. Students are expected to rewrite their assignments, emphasizing grammar, 
syntax, punctuation and style. 
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Table 1 





High School Course Content 


College Freshmen Course Content 


Composition 


•Course emphasis 


•Course emphasis 




-critical writing skills: argument, 
persuasion, rationale, thesis 
development. 


-critical writing skills: reason, persuasive 
argumentation, synthesis of ideas, thesis 
development. 




-creative writing deriving from readings. 


-combine reading and writing as forum of 
communication. 




-process: revisions/rewrites, editing, 
grammar, self-criticism, usage, 
punctuation. 


-process: revisions /editing, self-criticism, 
grammar, style. 




•Expected performance: 2 essays, 4 shorter 
papers. 


•Expected performance: four essays (3 subject 
related, 1 reflective). 


Algebra 


•Pre-calculus emphasis 


•Pre-calculus emphasis 




-graphing, polynomial functions, rational 
functions, exponential functions, 
logarithms. 


-graph sketching, functions, 
linear/nonlinear, exponential functions, 
logarithms. 




•Expected performance: weekly quizzes, 
final. 


•Expected performance: weekly 

exams/ worksheets, midterm, final. 


Biology 


•Introduction to biology emphasis. 


•Introduction to biology emphasis 




-genetics, cell structure, mitosis, protein 
synthesis, photosynthesis, meiosis. 


-genetics, biological kingdoms, cell 
structure, mitosis, scientific methods, 
protein synthesis, photosynthesis, 
meiosis, sex linkage/genetics/polygenic 
inheritance. 




•Expected performance: journal, labs, paper, 
quizzes, final. 


•Expected Performance: weekly 

labs/workshops, quizzes, papers, 
midterm, final. 


Spanish 

Language 


•Introductory course 

-speaking, reading, writing, some cultural 
studies. 


•Introductory course 

-speaking, reading, writing, some cultural 
studies. 




-basic grammar, usage (feminine, 
masculine), pronunciation, vocabulary. 

•Expected performance: workbooks, quizzes, 
paper, oral exams. 


-basic grammar, usage (feminine, 
masculine), syntax, pronunciation, 
vocabulary. 

•Expected performance: written exams, 
quizzes, labs, oral exams. 



Other courses shared similar, or even greater levels of redundancy than found in 
Composition. For example, College Algebra covers virtually the same course content as the 
high school Algebra II course, including: functions, graphing, polynomials, and equations 
and inequalities. Similarly, college and high school introductory Spanish language courses 
are virtually indistinguishable, each focusing on basic elements of grammar, pronunciation, 
and vocabulary. Additionally, although college level introductory biology does add some 
content to high school science courses, much of what is covered at the college level will be 
review for many freshmen students, including: cell structure, mitosis, basic proteins, 
photosynthesis, and meiosis. In sum, our content analysis of college and high school course 
syllabi exposed substantial redundancy in the levels of knowledge, skills, and abilities 
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expected of high school and college freshmen students. In other words, college freshmen 
aren't expected to know or do much more than they did as high school juniors and seniors. 

So, what's the problem with the redundancy between high school and college level 
coursework? At first glance, the knowledge and skills taught at both levels appear important 
enough to warrant some overlap, particularly if the college level coursework adds some 
additional material to what students learned in high school. But our content analysis of 
high school and college level course syllabi produced so much redundancy that it is clear 
that many students in college courses are being taught almost exactly what they were taught 
in their high school courses, and not surprisingly, are earning good grades. In light of the 
course redundancy, how well do college freshmen grades indicate student readiness for 
what should be more rigorous coursework? Not very well, unless as educators, we're 
satisfied with many of our college freshmen reviewing what the they were taught in high 
school. 

The problem of course redundancy, coupled with the increasing levels of 
remediation required at OSSHE institutions (discussed above), helps to suggest a weakness 
in current higher education admission and placement policies. Even college placement 
exams don’t remedy the problem, and as one Teaching Assistant from an OSSHE institution 
put it, "a lot of what we do will be review for many of our students, but a lot of it will be too 
much for some students. Even with placement tests, we can't determine how much each 
student really knows until we get them in here." Better determining "how much each 
student knows" is at the heart of the dilemma, and is the essential purpose of the PASS 
project. PASS allows for a more stronger, more accurate assessment of what students' skills 
and abilities are, and at the same time, provides clearly defined standards and meaningful 
assessment criteria. In this way, PASS helps to reduce the variance present in each incoming 
freshman class. In doing so, it provides college instructors better information on student 
readiness for college level coursework. 

Decreasing Value of High School Grade Point Averages as Predictors of College Readiness 

We have discussed why, as a result of grade inflation, high school GPAs simply may 
not provide the same measure of college readiness that they once did. We have also 
demonstrated that grade inflation and redundancy in course content have undermined the 
value of freshmen college grades as true measures of what students actually know. In this 
subsection we continue exploring these issues by examining the relationship of high school 
GPA and freshman performance from another perspective. Specifically, our research 
suggests that high school GPAs may accurately predict college readiness only for those 
students with relatively high and relatively low GPAs; for the vast majority of entering 
freshmen the GPA is not a highly accurate predictor of freshman year grades. 

Data on college freshmen who entered an Oregon State System of Higher Education 
(OSSHE) institution in Fall term, 1994 (the last year for which this data is currently 
available), were used to explore the assumption that high school GPAs predict freshman 
performance. If high school GPAs are good predictors of student readiness for freshman- 
level college work, we would expect to find a reasonably close correlation between high 
school and college GPAs. In other words, if a student enters college with a very high GPA, 
we would generally expect that student to do well in the freshman year, particularly given 
the overlap in curriculum we identified earlier. Likewise, if a student comes to college with 
a low GPA we might expect that student to have a relatively low college GPA. 

We compared the high school GPAs of entering freshmen with their GPAs after the 
first year of college. Essentially, we "tracked" the performance of all freshmen in the first 
year of courses at all OSSHE institutions, and compared that performance to their high 
school grades. What we found was that for the vast majority of freshmen, high school GPAs 



are not accurate predictors of how well or poorly they will perform in their first year of 
college. 

In Fall term, 1994, 8,206 freshmen entered OSSHE institutions. Of those freshmen, 
1,094 (15%) entered with a high school GPA of between 3.75 and 4.00 (on a 4.00 scale). For 
these students, high school GPAs did seem to predict college readiness, as nearly 70% of 
them finished their first year of college with GPAs in the top 15% of all OSSHE freshmen. 
Similarly, 1,052 (15%) students entered college with high school GPAs of 2.74 or below. 
Again, for these students high school GPAs seem to correlate closely with college 
performance, as 74% finished their freshmen year with GPAs in the bottom 15% of all 
OSSHE freshmen. What this tells us is that if a student enters college with a high school 
GPA in the top or bottom 15% of the entering class, they are very likely to finish their first 
year of college in the same stratum. But that only accounts for 30% of the students — what 
about the other 70% who enter college with high school GPAs somewhere in the middle, 
between 2.75 and 3.74? 

The data show that for this population of students, high school GPAs do not closely 
correlate with performance in freshman courses. Graph 5, below, sets out the distribution of 
freshman grades for all OSSHE students who entered with high school GPAs between 2.75 
and 3.74 (on a 4.00 scale). The shaded areas in each bar in the graph indicate different strata 
of freshman GPAs, while the bars indicate different strata of high school GPAs. Thus, the 
graph shows that for those students entering OSSHE with high school GPAs between 3.50 
and 3.74, 10% completed their freshmen year in college with GPAs between 3.75 and 4.00. 
Similarly, as the graph shows, 10% of all freshmen who entered OSSHE with GPAs between 
3.25 and 3.49 finished their first year of college with GPAs of 3.75 or higher. 



Graph 5 



High School GPA by College Freshmen GPA for Students Entering OSSHE with GPAs between 2.74 and 3.75 




I 3.75-4.00 



3.5-3.74 

3.25-3.49 

3.00-3.24 



■ 2.75-2.99 
2.74 and Below 




Graph 5 illustrates the wide range of performance among OSSHE freshmen. If high 
school GPAs were truly accurate predictors of college readiness, we could reasonably expect 
to find a correlation between high school and college freshmen performance. For example, 
21% of students entering OSSHE with high school GPAs between 3.00 and 3.24 ended their 
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freshmen year with college GPAs between 3.00 and 3.24. The same percentage (21%) of 
OSSHE freshmen entering with high school GPAs of 3.5 to 3.74 completed their freshmen 
year of college with GPAs between 3.00 and 3.24 — the same level of performance as those 
students entering college with substantially lower high school GPAs. This similarity in 
college performance is true for both relatively high college GPAs and low college GPAs. For 
instance, a student entering OSSHE with a high school GPA between 3.50 and 3.74 is just as 
likely to have a college freshmen GPA between 2.75 and 2.99 as the student who enters with 
a high school GPA between 3.00 and 3.24. 

What these data show is that high school GPAs between 2.75 and 3.74 — which 
includes the vast majority of entering freshmen — don't predict freshman performance very 
well. It is true that high school GPAs seem to predict the college success of students with 
GPAs falling into extreme strata (the top and bottom 15%), but we expect that a certain 
number of students will perform very well or very poorly irrespective of assessment 
methods. 

This subsection presented data from a single year of one state system of higher 
education. It is not intended to be an exhaustive, longitudinal study of high school GPAs as 
predictors of college readiness. Rather, the data are presented to provoke dialogue and raise 
some questions regarding the role of high school GPA in higher education admissions. 

The preceding summary of research conducted by the PASS project brings into 
question many of the traditional assumptions regarding college readiness. Although the 
data were derived from a single state system (OSSHE), we have no reason to believe that 
Oregon is a particularly unique context for exploring such issues. At a minimum, we have 
shown that there are reasons to question some of the assumptions regarding higher 
education admissions practices. Evidence from a number of areas including high levels of 
remediation in writing and math required for college freshmen; the inflation of high school 
grade point averages; the inflation of freshmen grades; redundancy in high school and 
college level coursework; and the decreasing value of high school grade point averages as 
predictors of readiness in college level coursework all contribute to this conclusion. 

Section 6: Major Issues and Next Steps’ 

The project still has a significant number of issues to address. These include the 
following: 

Assessment issues 

Designing an entirely new assessment system is daunting. However, PASS will draw 
upon several sources of expertise. The Oregon Department of Education will let contracts for 
development of criterion-referenced tests that match up with the six PASS content areas. 
Several national organizations such as New Standards, the Center for Research on 
Evaluation, Standards, and Student Testing, and American College Testing all are providing 
assistance or materials for development of Common Assessment Tasks. National 
consultants with experience in developing performance-based scoring guides are assisting in 
development of the standards for Teacher Verifications. 

The assessment system will be developed in stages to ensure each element meets 
technical adequacy standards and yields students who demonstrate success in college. An 
ambitious plan for assessment such as this will work only if the data generated serve several 
purposes. These data will be used for CIM, CAM, and PASS, and will functionally replace 



‘ This section appeared in an edited form in the document: Responding to School Reform: Higher Education 
Defines New Roles in Oregon. Wisconsin and Florida. November, 1996, Education Commission of the States, 
'07-17th St., Suite 200, Denver, Colorado 80202-3427, 303-299-3600. 
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grades. Teachers will adapt their grading practices to this system, not add it on top of current 
practices. 

Finding the time for all this assessment will require high schools to designate some 
days as "assessment days," in much the same way they now establish final examination 
schedules. Common Assessment Tasks, such as a foreign language demonstration, will 
have the option of being videotaped and sent to another teacher or school for scoring. Some 
teachers may receive some additional time to conduct scoring. 

Articulation issues 

Oregon community colleges are very interested in PASS proficiencies. They recognize 
that if community colleges have no standards, students might tend to avoid PASS standards 
in high school, then go to the community college with the idea of transferring to a four-year 
school after a minimal amount of coursework. Community colleges are not interested in 
becoming a "back door" to higher education, so they are developing their own standards. 
Although the community college as a whole will remain an open-door institution, specific 
programs may have entrance requirements. The college transfer program would have as its 
entrance requirement that students have met PASS proficiencies. 

The other natural area of conversation between community colleges and four-year 
institutions is the transfer program. As OSSHE enters into the next round of renegotiating 
common course numbering with community colleges, initial attempts at thinking about 
transfer in terms of proficiency will be initiated. Clearly, it will take some time to move to 
such a system. However, these early efforts demonstrate a commitment to a more 
"seamless" educational system where students move between institutions and levels based 
on proficiencies that are common across institutions. 

Funding issues 

The development work is currently being funded by three federal grants from the 
U.S. Department of Education and one grant from the Pew Charitable Trusts. In addition, 
OSSHE is supporting the project financially. The purposes and amounts of the grants are as 
follows: 



• Department of Education, Fund for the Improvement of Post- Secondary Education: To 
support development of assessments and 20 partnership high schools, $300,000 

• Department of Education, Office of Educational Research and Improvement, Fund for 
Innovation in Education: To support development of training materials, trainers, and to 
expand the number of partnerships, $1,100,000 

• Department of Education, Office of Reform Assistance and Dissemination, Eisenhower 
Professional Development Federal Activities Program: To develop materials and 
strategies for teacher education programs to better prepare teachers to teach in standards- 
based schools, $1,200,000 

• The Pew Charitable Trusts, Philadelphia, Pennsylvania: To implement PASS standards 
in all 12 high schools in the Portland Public Schools, $600,000 

This system will have real costs beyond these one-time development expenses. 
Performance assessments will need to be updated regularly. Scoring materials will have to 
be printed and distributed widely to teachers. Regular opportunities for training will have to 
be available. 
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OSSHE does not intend to fund all the costs high schools would incur to train 
teachers or to conduct assessments. This is consistent with current policies. However, PASS 
will provide the materials necessary to operate the system and to train educators in its use. 
The partnership high schools will serve as models of how to implement PASS. In addition. 



a network of Key Trainers is being created, individuals who are skilled in staff development 
who will be available for schools to employ to train teachers in performance assessment 
generally and PASS specifically. Many districts are already designating individuals they 
want included in the Key Trainers pool. PASS will work closely with the Oregon 
Department of Education as it develops training for the CIM and CAM. 

PASS is currently working to establish the likely ongoing costs of the system. An 
estimate will be prepared for submission to the next session of the Legislature in 1997. Much 
of the work can be accomplished with participation from school districts and higher 
education institutions on a contributed basis. This system cannot have a high cost if it is to 
survive politically. Therefore, creativity and adaptation will be required throughout the 
development of assessment and training systems. 

The intent is to have the same staff in admissions offices. However, these individuals 
will spend their time in somewhat different ways. They will still receive numerical 
summaries, the proficiency average, but will have access to more high-quality supplemental 
data. Such data can be used for placement and advanced standing purposes, as well. 
Admissions officers will work more closely with faculty to place students more 
appropriately as they enter college. 

Equity and access issues 

The goal is not to make college admission harder or easier, but to ensure that 
students are better prepared to succeed once they are admitted. For this to happen, students 
will need to be more knowledgeable and skillful academically than they are currently. 

However, since this higher level of preparation is being expected of all Oregon 
applicants, students will have the same relative likelihood of being accepted for admission. 
PASS will work with high schools to set standards at a level that is higher than current 
expectations, but achievable by the vast majority of students who currently strive for college 
admission. These students may have to work more diligently to meet proficiency standards, 
but those who do so will be just as likely to be admitted as they would under the current 
system. 

Furthermore, requirements for the CIM are designed to result in more students 
meeting high performance standards. If students are not reaching CIM performance levels, 
proficiency scores required for admission will be adjusted accordingly. 

OSSHE is committed to examine continuously the performance of all students 
admitted via proficiency demonstrations to determine if some PASS requirements have 
differential effects on minorities, women, non-native speakers, and students with special 
learning needs. The system will be thoroughly reviewed and modified on a regular basis, 
while retaining high standards for all students. OSSHE's goal is to make PASS as equitable 
as possible. 

The broader issue is whether a standards-based system is more equitable than the 
current one where standards are unstated and largely determined by teachers in isolation. 
Evidence suggests that students at inner city schools who receive A's perform about as well 
in college as their counterparts in suburban districts who receive C's. When standards are 
explicit and assessments consistent, communities can easily compare their students' 
performance. Furthermore, parents and community activists, armed with proficiencies and 
samples of student work, can hold educators accountable in much more specific ways than 
they can now. Standards are not a panacea, but they offer perhaps the best hope for 
underserved populations to gain access to high-quality educational experiences. At the same 
time OSSHE will continue efforts to ensure the number of historically underrepresented 
students increases consistently. A proficiency-based admission system should help increase 



the pool of eligible applicants and improve the rate at which they complete a baccalaureate 
program successfully. 

Political issues 

School reform exists in a social and political context. PASS benefits because it is a 
creation of the State Board of Higher Education, not the Legislature. PASS has more 
flexibility and can adapt to changes in legislation. However, PASS is primarily an attempt to 
link to education reforms occurring in high school, to guide and support those reforms 
toward the goal of improve college preparation. The intent is not to control those reforms. 
PASS, therefore, has a delicate relationship with the CIM and CAM. As they are developed, 
PASS seeks to influence them but not dominate them. The CIM in particular is a general 
education certificate that contains areas of study not necessarily appropriate for college 
admission. PASS does not seek to have the CIM address college preparation only, but to 
ensure that students who receive CIMs can be college-ready. This leads to an interactive 
development process where each player moves its performance-based system along another 
step, then recalibrates it with the work of the other. This may not be a perfect process, but it 
does represent the most substantive and sustained collaboration between K-12 and higher 
education systems in memory. 

The long-term viability and direction of education reform 

Questions remain regarding how enduring education reform efforts generally and 
the movement toward standards specifically will be. Each wave of reform over the past 15 
years has led to a more sustained and intensive look at schooling in America. This level of 
energy and scrutiny cannot continue indefinitely. However, evidence suggests that the next 
target for examination and calls for reform will be higher education. Perhaps the efforts 
being undertaken in Oregon to develop close ties with public schools and to establish 
standards that result in improved student preparation for college will be a precursor, not 
just of high school reform, but of a broader call for changes in higher education as well. 

Implications for high schools 

When PASS is fully implemented high schools will have both the opportunity and 
challenge to do things differently. Removing subject area requirements based on course title 
and time opens the door to more variety in instructional formats. Courses need not meet 
for a specified number of minutes per week, nor do they necessarily have to be designated 
"college-bound." Students with a variety of educational objectives can learn together, 
provided assignments and assessments are adapted accordingly, with college-bound 
students completing PASS-required assessments and meeting PASS proficiency 
requirements. 

A proficiency-based system suggests more students will take the same challenging 
curriculum, and that instruction, not curriculum, will vary greatly. A core curriculum 
means more than identifying several classes as core. It means designing a core set of 
learnings that enable students to reach high standards, in this case standards that lead to 
college admission. 

The trade-off can be that students have less time for electives as stand-alone learning 
activities. Such activities can often be incorporated into core curriculum to some degree, but 
if the core is large, students have less time for "electives." However, in a proficiency-based 
system, colleges make no distinction between required and elective courses. What is 
important is what the student learns or masters in each class. Traditional electives will be 
challenged to demonstrate how they link to PASS proficiencies. When students perceive 
the link, they will be more likely to enroll in them. 



The role of counselors can be expected to change. They will do more than count 
credits and provide letters of recommendation. While their role in a proficiency based 
system is still emerging, early indications are that they help teachers and students 
understand better what the standards are and they help them assemble supplementary 
evidence of learning that accompanies the proficiency transcript. Since the colleges, not the 
high school, set the standards and requirements, less interpretation is needed from the 
school. Furthermore, personal references may still be useful, but will be less influential. It's 
not who you know, but what you know. 

High schools may be challenged by service providers who can help prepare students 
for proficiency demonstrations. Similarly, home schoolers may have an easier time of it if 
their students no longer need a high school diploma, but can simply meet proficiency 
requirements and demonstrate this to a credible authority, such as a high school. 
Competition is not assured, but is perhaps enabled slightly when the standards for college 
admission are public and accessible. 

A proficiency-based system acknowledges learning no matter where it occurs. This 
means students who learn, say, a foreign language outside of school could demonstrate 
proficiency and not need to take a class in high school. Such students might accelerate 
through the high school program, or reach higher levels of achievement while remaining 
in high school and receive college credit. High schools will have to offer more than simply a 
diploma and transcript if they are to retain students in a performance-based environment. 

Implications for higher education 

If high schools are potentially affected by proficiency-based admission, so are colleges 
and universities. A proficiency-based system can be expected to produce more qualified 
students who are used to learning and being assessed in somewhat different ways than 
today's students. 

Students will be expected to be at higher levels of preparation in areas such as foreign 
language and mathematics. This means fewer introductory courses taught by teaching 
assistants, or, at the least, higher expectations of teaching assistants or more instruction by 
professors. Some departments may need to redesign their introductory courses and 
subsequently their sequence based on the proficiencies students have mastered to be 
admitted. 

Many colleges will be challenged if a larger proportion of students proceed to the 
junior level and seek to enter a master. If colleges are admitting more students who are 
fully qualified, and more students who may already have received credit for some 
introductory courses, more students will proceed more rapidly to pursue majors. Most 
campuses do not have enough spaces in majors to accommodate all students whom they 
admit as freshmen. 

Similarly, colleges should plan to offer less remediation on campus. Students should 
not receive the message that they need not be proficient once they arrive on campus. If 
proficiency is certified in high school, students will not engage in remedial courses, nor 
should they expect to have them available. Some students may legitimately require 
additional academic support, but colleges in Oregon will no longer assume a large 
proportion of the student population is incapable of doing college-level work upon 
admission. The money saved will be reallocated to academic programs in ways that support 
proficiency-based learning. 

One example is credit by demonstration. Students will have more opportunities to 
meet program requirements by completing a task or performance. Students will pay for 
these credits at the same rate they would if they were to sit in a classroom. Departments will 



develop procedures to score student demonstrations or projects, but will not be called upon 
to offer as many classes. This approach will succeed because students will be much more 
comfortable with this approach to demonstrating proficiency, having had similar 
experiences already in high school. They will be familiar with scoring guides, portfolios, and 
demonstrations. More students will also earn credits by applying learning in context 
through field experiences, internships, and projects. These strategies enable increased 
productivity, as the same number of faculty accommodate more students without 
increasing faculty workload appreciably. 

Colleges and universities can expect to have more discerning students who expect 
more from their education. Oregon's public colleges and universities are taking steps to 
accommodate these learners by expanding the number of courses that have extensive 
student involvement, increasing the use of technology on campus as a way to demonstrate 
proficiency, and developing new modes of assessment at the "junior rising" and 
baccalaureate levels. Once proficiencies are accepted as a way to determine admission, the 
door is open to expecting some proficiency to earn a diploma. OSSHE is anticipating these 
issues as it institutes proficiency-based admissions. 

Oregon has embarked on a journey toward a new and largely unexplored method of 
student admission. Its goal initially was to be a participant in school reform, rather than a 
spectator. As the project has developed, higher education realized a proficiency-based 
approach could serve a number of beneficial purposes beyond the original intent. This 
multidimensional utility encouraged development work to continue. As work continued, it 
became clear that OSSHE's work on PASS was beginning to shape high schools' 
interpretation of school reform. The PASS proficiency system provided needed clarity and 
definition. The rationale and reward for pursuing the PASS proficiencies was also self- 
evident. Currently, development work is reaching the challenging point of creating the 
assessment system and performance levels necessary to specify what will actually be 
expected of students. More work remains in key areas like transcripting, transferring 
students across state lines, training teachers, and involving higher education faculty more 
broadly. At this point PASS is an ambitious "proof of concept" project that is demonstrating 
it is feasible to admit students based on performance. The more challenging work of scaling 
up and institutionalizing the system lies ahead. 



Section 7: PASS Project Evaluation: Summary of Major Findings 

Year 1 Findings 

The PASS project evaluation was organized as four strands of research: Formative 

evaluation of 1) major PASS project activities and 2) Lead Assessors' development and use 
of PASS curriculum and assessment materials; and documentation of 3) school practices 
and student values and educational aspirations and 4) teachers' perception of the effects of 
PASS on student learning and achievement as a forerunner to a summative evaluation of 
student academic outcomes. Data sources were: Project activity feedback forms, on-site 
teacher interviews and classroom observations conducted throughout the fall and winter of 
1995-96 in the 12 original project schools, and teacher and student surveys administered in 
June in the same schools. The following subsections contain excerpts from the four areas 
listed above. 

PASS Implementation 
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PASS Lead Assessors are using PASS proficiencies in classes that represent a wide range 
of academic levels, 9 th and 10 th grade level as well as 11 th and 12 th grade level courses and 
general as well as college prep and AP program classes. 
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• Three quarters or more students who did PASS coursework viewed it very or somewhat 
favorably across several descriptors. The most frequently endorsed negative view was 
that the coursework lacked applicability to the real world. 

• Lead Assessors particularly math teachers are focusing curriculum content in order to 
accommodate PASS proficiencies. Future evaluations need to examine closely these 
curriculum trade-offs. 

• The use of PASS proficiencies has led Lead Assessors to assess a wider range of 
dimensions of student work than they have normally used. 

• Lead Assessors reported that their involvement in PASS development work for the 
most part had not strained their time and energy and on the contrary had enhanced their 
curricular and assessment strategies and even to some extent their instructional 
strategies and depth of subject area knowledge. 

• Most Lead Assessors collaborated with colleagues both inside and outside their school in 
developing PASS coursework. Lead Assessors of geographically remote schools 
especially appreciated the opportunity to exchange ideas with colleagues in other PASS 
partner schools. Future evaluations should address the question of how to increase Lead 
Assessor utilization of the PASS network since of Lead Assessors have not yet availed 
themselves of it. 

• Teacher interviews, surveys, and focus groups indicated that Lead Assessors' colleagues 
including administrators often show little interest or fail to provide needed support for 
PASS implementation. Gaining collegial cooperation is the primary concern of Lead 
Assessors at this point in their work. 

• A majority of teachers not involved in PASS believe PASS is not sufficiently aligned 
with CIM/CAM, and one-third of Lead Assessors hold the same view. 

School Practices and Student Attitudes 

• Differences in curricular and instructional practices between higher and lower academic 
level courses and across different subject areas appear pervasive and pronounced on the 
basis of both teacher and student self-report and classroom observations. However, 
classroom observations also revealed that highly innovative practices were in use in 
those classes where they were not typically found, for example, lower level math classes. 
Future evaluations should examine closely both whether the differences influence the 
implementation of PASS and conversely what the effect of PASS implementation is on 
these differences. 

• The college aspirations, work experience, and peer values of students in higher level 
academic courses differ markedly from students in lower academic courses. Future 
evaluations should examine the effect of broadened implementation of PASS on these 
differences. 

• Large percentages of students overall felt their jobs were more enjoyable than school and 
that attending class and studying were only somewhat (as opposed to very) important. 
Future evaluations should continue to monitor these attitudes in relation to broadened 
implementation of PASS. 

• Survey responses showed that teachers overall assign grades on the basis of factors that 
are both consistent with PASS, students' absolute level of achievement, for example, and 
inconsistent, class attendance, for example. Interestingly, focus group findings pointed 
out that Lead Assessors are having to overcome students' idea that effort, class 
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attendance, just turning something in, etc., should count for a great deal. The survey 
responses suggest that teachers reinforce this attitude through their grading practices. 

• Both teacher interviews and surveys indicated that school staff are eliminating remedial 
courses to heighten standards for achievement overall at the same time that some 
teachers are exerting pressures to create such courses as the only means to avoid higher 
failure rates. 

PASS Impact on Learning 

• Most Lead Assessors felt students' work on PASS proficiencies enhanced their learning 
in a range of areas. Future evaluations need to collect information that will enlarge 
understanding of why even small percentages of Lead Assessors felt PASS had a negative 
effect on student learning. 

• Lead Assessors' survey responses indicated that PASS had a more favorable impact on 
higher achieving students than lower achieving ones. Yet, focus group findings 
revealed that Lead Assessors felt both groups of students benefited from the more precise 
specification of learning goals they had accomplished with use of PASS proficiencies. 
Future evaluations need to continue to monitor the impact of PASS on students who 
have historically achieved at lower levels. 

Very Preliminary Findings from Year 2 

Curriculum 



• At a general level and for teachers involved in year 2 or 3 of the project, teachers are 
adapting their curriculum to meet proficiency standards (as opposed to adapting the 
proficiencies to fit their curriculum [more typical of year 1 participants]). After teachers 
have determined a match between one or more proficiencies in the class(es) they are 
teaching, they are using the proficiency to restructure curriculum. This has resulted in 
covering more material in some areas and covering less in other areas. 

• The issue of curriculum coverage concerned some teachers initially as they planned 
their learning activities for students; however, as students became more involved in the 
subject matter and more excited about the depth in which they were learning material, 
teachers' concerns abated. 

• The issue of curriculum coverage is of greatest concern for social studies teachers who 
feel they lack the direction they need to work with the proficiencies. Social studies 
teachers as a whole are having the greatest difficulty with PASS proficiencies. 

Instruction 



• For year 2 and 3 participants, instruction has been less directed by the teacher (their role 
is changing to that of a coach or a facilitator) and more directed by the students. 

Although there is still a lot of lecturing going on, teachers are also using a greater variety 
of instructional activities in their classrooms. Many teachers, for example, are 
developing projects in which students work collaboratively and many teachers are 
developing and using scoring guides or rubrics for students. 



ERjt 



According to students, projects give students the opportunity to apply the knowledge 
and skills they have acquired in a way that traditional assignments (i.e., worksheets) do 
not. Many students say that when their learning is self-directed and they can work on 
projects to demonstrate what they know, what they have learned will "stick with them 
for a long time." 
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• The use of scoring guides has had the most profound influence on students. When 
used as part of an instructional activity, scoring guides provide clear expectations for 
students For the first time for many students, they know what they need to do to 
complete an assignment, specifically what is important to include, how the information 
needs to be presented, and the criteria that will be used to make a judgment (i.e., grade). 

• Teachers, while recognizing the instructional value of scoring guides, say that they do 
not have adequate time to produce them for all of the learning activities. 

Assessment 

• Assessment activities are of the greatest concern for the majority of PASS participants. 
Concerns include having students produce a sufficient body of work, to organizing the 
logistics and management of the assessment activity (e.g., "blind" evaluations for oral 
language proficiency), to trusting colleagues to review bodies of work, to using the 
ratings to make decisions about grades, to recording and reporting assessment results and 
meeting proficiencies. 

• Math teachers are also concerned that there needs to be a test to objectively assess if a 
student has met a proficiency. 

• Despite these concerns, second and third-year teachers are using assessment results to 
revise their curriculum and instruction. 

• As teachers are becoming more confident of the assessment results it is conceivable that 
they will use assessment activities to frame curriculum and instruction activities. 

Collaboration 

• Although teachers are the first to acknowledge the value of collaboration in a project like 
PASS, there is very little collaboration around PASS requirements going on within 
schools and across schools. 

• Many teachers within schools are experiencing frustrations in areas that teachers in 
other schools have tackled successfully (e.g., using a math proficiency in a "non-math" 
class such as electrical engineering; developing scoring guides) and information is not 
being shared. 
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